Javier Estrada
Michael Underwood
Elizabeth Subject-Scott
X are the completely observed variables.
Y are the partly missing variables.
Z is the component of the cause of missingness unrelated to X and Y.
R is the missingness.
When type is MCAR and the amount of missing data is small, deletion can be used.
2 Types
Listwise deletion occurs when the entire observation is removed.
Pairwise deletion occurs when the variable of an observation is removed.
Deleting missing data can lead to the loss of important information regarding your dataset and is not recommended.
Imputation
2 Types
Single Imputation
Multiple Imputation
Methods include:
Rubin’s Rules: Average the estimates across m estimates.
Calculate the standard errors and variance of m estimates.
Combine using an adjustment term (1+1/m).
Regression Imputation is based on a linear regression model. Missing data is randomly drawn from a conditional distribution when variables are continuous and from a logistic regression model when they are categorical.
Predictive Mean Matching is also based on a linear regression model. The approach is the same as regression imputation except instead of random draws from a conditional distribution, missing values are based on predicted values of the outcome variable.
Hot Deck (HD) imputation is when a missing value is replaced by an observed response of a similar unit, also known as the donor. It can be either random or deterministic, which is based on a metric or value. It does not rely on model fitting.
Stochastic Regression (SR) Imputation is an extension of regression imputation. The process is the same but a residual term from the normal distribution of the regression of the predictor outcome is added to the imputed value. This maintains the variability of the data.
According to Rubin’s Rule, in multiple imputation m imputed values are created for each of the missing data and result in M complete datasets. For each of the M datasets, an estimate of \(\theta\) is acquired. Let \({\hat{\theta}}_{m}\) and \({\hat{\phi}}_{m}\) be an estimator of the variance of \({\hat{\theta}}_{m}\) based on the Mth complete dataset.
The combined estimator of \(\theta\) is given by:
\[{\hat{\theta}}_{M} = \displaystyle \frac{1}{M}\sum_{m = 1}^{M} {\hat{\theta}}_{m}\]
The proposed variance estimator of \({\hat{\theta}}_{M}\) is given by:
\[{\hat{\Phi}}_{M} = {\overline{\phi}}_{M} + (1+\displaystyle \frac{1}{M})B_{M}\]
with a correction factor of \((1+\displaystyle \frac{1}{M})\),
where the average within imputation variance is (normal standard error):
\[{\overline{\phi}}_{M} = \displaystyle \frac{1}{M}\sum_{m = 1}^{M}{\hat{\phi}}_m\]
and the between imputation variance is (how much error because of imputation process):
\[B_{M} = \displaystyle \frac{1}{M-1}\sum_{m = 1}^{M}({\hat{\theta}}_{m}-{\overline{\theta}}_{M})^{2}\]
Observed data follow a multivariate normal distribution.
Missing data are classified as MAR, which is the probability that a missing value depends only on observed values and not unobserved values.
The parameters \({\theta}\) of the data model and the parameters \({\phi}\) of the model for the missing values are distinct. That is, knowing the values of \({\theta}\) does not provide any information about \({\phi}\).
Step 1: Impute missing data
Step 2: Run regression models on all imputation sets
Step 3: Pooling regression results into one regression result
MICE = Multivariate Imputation by Chained Equations.
mice() method in R.
The method creates multiple imputations (replacement values) for multivariate missing data.
The MICE algorithm can impute mixes of continuous, binary, unordered categorical and ordered categorical data.
This can be doing using the imputation regressions of the mice() method such as:
predictive mean matching (numeric data)
logistic regression imputation (binary data with two factorial levels)
polytomous regression imputation for unordered categorical data (factor > 2 levels)
proportional odds model for (ordered, > 2 levels).
King County, Seattle Home Sale Prices between 2014 and 2015
Contains the sale prices of 21,613 houses
The original dataset contained 21 columns with various selling attributes.
For the purpose of this project, we have condensed the variables to the following 4:
# laod library
library(mice, warn.conflicts=FALSE)
# load data
original = read.csv("kc_house_data.csv")
#first 5 rows
head(original, 1) price bedrooms bathrooms sqft_living
1 221900 3 1 1180
# duplicate dataset
house <- original
# replace some values with NA to create missing data
set.seed(10)
house[sample(1:nrow(house), 200), "sqft_living"] <- NA
house[sample(1:nrow(house), 200), "bedrooms"] <- NA
house[sample(1:nrow(house), 200), "bathrooms"] <- NA
house[sample(1:nrow(house), 100), "price"] <- NA
# check for missing data
sapply(house, function(x) sum(is.na(x))) price bedrooms bathrooms sqft_living
100 200 200 200
Our dataset now has 700 NA/missing values. This equates to about 3% of the data (700/21,613).
Variables sorted by number of missings:
Variable Count
bedrooms 0.009253690
bathrooms 0.009253690
sqft_living 0.009253690
price 0.004626845
The red box plot on the left shows the distribution of sqft_living with price missing. The blue box plot shows the distribution of the remaining data points. The same is true for the price box plots on the bottom. We expect the red and blue boxplots to be very similar if our assumptions of MCAR data are true. In this case, they are.
Since we are missing about 3% of our data, we need to perform at least 3 imputations. This will be done using the mice() function:
Since 5 is the default, we will use that (the m parameter can be used to adjust the the # of imputations)
The set.seed will be given the value 1337 (any number can be used here) to retrieve the same results each time the multiple imputation is performed.
iter imp variable
1 1 price bedrooms bathrooms sqft_living
1 2 price bedrooms bathrooms sqft_living
1 3 price bedrooms bathrooms sqft_living
1 4 price bedrooms bathrooms sqft_living
1 5 price bedrooms bathrooms sqft_living
2 1 price bedrooms bathrooms sqft_living
2 2 price bedrooms bathrooms sqft_living
2 3 price bedrooms bathrooms sqft_living
2 4 price bedrooms bathrooms sqft_living
2 5 price bedrooms bathrooms sqft_living
3 1 price bedrooms bathrooms sqft_living
3 2 price bedrooms bathrooms sqft_living
3 3 price bedrooms bathrooms sqft_living
3 4 price bedrooms bathrooms sqft_living
3 5 price bedrooms bathrooms sqft_living
4 1 price bedrooms bathrooms sqft_living
4 2 price bedrooms bathrooms sqft_living
4 3 price bedrooms bathrooms sqft_living
4 4 price bedrooms bathrooms sqft_living
4 5 price bedrooms bathrooms sqft_living
5 1 price bedrooms bathrooms sqft_living
5 2 price bedrooms bathrooms sqft_living
5 3 price bedrooms bathrooms sqft_living
5 4 price bedrooms bathrooms sqft_living
5 5 price bedrooms bathrooms sqft_living